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1.0  EXECUTIVE  SUMMARY 


1.1  BACKGROUND 

The  Chemistry  Division  of  the  Naval  Research  Laboratory  (NRL)  has  participated  in  several 
programs  funded  by  the  Strategic  Environmental  Research  and  Development  Program  (SERDP) 
and  Environmental  Security  Technology  Certification  Program  (ESTCP)  whose  goal  has  been 
to  enhance  the  discrimination  ability  of  the  Multisensor  Towed  Array  Detection  System 
(MTADS).  The  process  has  been  based  on  making  use  of  both  the  location  information  inherent 
in  an  item’s  magnetometry  response  and  the  shape  and  size  of  information  inherent  in  the 
response  to  the  time-domain  electromagnetic  induction  (EMI)  sensors  in  either  a  cooperative  or 
joint  inversion.  In  all  of  these  efforts,  classification  performance  has  been  limited  by  the 
information  available  from  the  EMI  sensor.  Prior  to  this  project,  the  industry-standard  EMI 
sensor,  the  Geonics  EM61,  was  used.  The  EM61  is  a  time-domain  instrument  with  either  a 
single  time  gate  to  sample  the  amplitude  of  the  decaying  signal  or  four  time  gates  sampling 
relatively  early  in  time. 

To  make  further  progress  on  unexploded  ordnance  (UXO)  classification,  a  sensor  with  more 
information  available  was  required.  The  Geophex  GEM-3  sensor  is  a  frequency-domain  sensor 
with  up  to  10  frequencies  available  for  simultaneous  measurement  of  the  in-phase  and  quadrature 
response  of  the  target.  In  principle,  there  is  much  more  information  available  from  this  sensor 
for  use  in  UXO  classification  decisions.  The  commercial  GEM-3  sensor  is  a  handheld 
instrument  not  very  amenable  to  rapid,  wide  area  surveys.  This  project  was  funded  to  overcome 
this  limitation  through  the  integration  of  an  array  of  GEM-3  sensors  with  the  MTADS  platfonn. 

1.2  OBJECTIVES  OF  THE  DEMONSTRATION 

The  objective  of  this  project  was  to  demonstrate  the  optimum  system  that  delivers  the  most  UXO 
classification  performance  possible  while  retaining  acceptable  survey  efficiency.  Working  with 
a  modified  GEM-3  sensor,  we  have  designed  a  three-sensor  array  and  demonstrated  it  at  the 
Standardized  UXO  Demonstration  sites  at  Aberdeen  Proving  Ground  (APG)  and  Yuma  Proving 
Ground  (YPG).  For  the  Blind  Test  Grid  and  the  Open  Field  Areas,  our  ranked  target  picks  were 
submitted  to  the  Aberdeen  Test  Center  (ATC)  for  scoring.  These  scoring  results  are  the  basis  for 
accessing  the  success  of  the  demonstrations. 

1.3  DEMONSTRATION  RESULTS 

The  GEM-3  sensor  was  selected  for  this  towed  array  because  of  its  performance  at  the  JPG-IV 
demonstration  in  1998.  With  the  capability  of  measuring  in-phase  and  quadrature  signals  at  10 
frequencies  over  the  30  hertz  (Hz)  to  20  kilohertz  (kHz)  range,  the  GEM-3  should  have  the 
necessary  information  to  enhance  UXO  discrimination  from  clutter  more  effectively  than  the 
current  state  of  the  art.  Overall,  the  results  from  the  GEM-3  array  described  in  this  report  at  the 
Standardized  UXO  Demonstration  Sites  were  a  disappointment. 

The  GEM-3  array  was  capable  of  detecting  ordnance  to  reasonable  burial  depths.  All  ordnance 
was  detected  on  the  APG  Blind  Grid  to  burial  depths  equal  to  1 1  times  the  UXO  item’s  diameter. 
It  was  not  effective  at  distinguishing  UXO  from  clutter.  Even  after  adjusting  for  items  emplaced 
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too  deep  to  be  detected,  high  discrimination  probabilities  of  detection  (Pd)  were  achieved  only  at 
high  levels  of  probabilities  of  false  alarm  (Pfa). 

The  primary  cause  is  an  issue  of  signal-to-noise  ratio  (SNR).  Model-based,  dipole  inversions 
require  fairly  high  levels  of  SNR  (>10)  to  return  reliable  results.  For  the  three  test  sites  visited  by 
the  system,  the  average  noise  levels  varied  from  less  than  one  part-per-million  (ppm)  for  mid¬ 
frequency  quadrature  responses  to  almost  10  ppm  for  the  in-phase  response  at  all  frequencies. 
The  signal  level  for  most  of  the  emplaced  ordnance  was  in  the  range  of  1  to  10’s  of  ppm.  This 
response  level  is  too  weak  to  invert  reliable  ordnance  signatures  across  all  the  frequencies  for 
both  the  in-phase  and  quadrature  responses.  Distinguishing  ordnance  from  clutter  relies  on 
accurate  inversion  of  the  in-phase/quadrature  signature  and  the  comparison  of  this  signature  to 
the  library  of  expected  ordnance.  At  low  SNR,  this  is  not  possible. 

Further  complicating  this  library  comparison  process  is  the  large  number  and  variety  of  UXO 
items  emplaced  at  the  test  sites.  Given  the  error  bars  on  the  inverted  parameters,  there  is 
necessarily  a  large  overlap  between  UXO  and  clutter  signatures. 

It  has  also  been  noted  that  good  inversion  results  from  dynamic  survey  data  require  accurate 
positioning  of  the  sensors,  on  the  order  of  sub-centimeter  positioning.  The  positioning  and 
orientation  of  the  GEM-3  array  was  provided  by  a  cm-level  Global  Positioning  System  (GPS). 

1.4  IMPLEMENTATION  ISSUES 

Implementation  of  the  methods  used  in  this  demonstration  requires  additional  survey  time 
compared  to  a  minimum-effort  detection  survey.  We  have  shown  that  the  MTADS  can  detect  all 
UXO  of  gauges  40  mm  or  larger  with  a  total-field  magnetometry  survey  alone.  For  ordnance 
target  sets  that  include  60-  and  81 -mm  mortars  at  depths  of  0.75  to  1  m  and/or  20-  and  30-mm 
submunitions,  an  overlapping  EMI  survey  is  required  to  generate  sufficient  data  density  and  to 
get  a  high  detection  probability.  This  increases  the  survey  hours  on  site  although  it  does  not 
impact  the  mobilization  and  data  analysis  costs.  In  many  cases,  the  extra  survey  costs  are  only 
equivalent  to  the  cost  of  digging  one  or  two  additional  targets  per  acre. 

The  MTADS  GEM-3  array  is  a  vehicular-towed  array  with  the  operating  limitations  of  a  towed 
system.  Support  requirements  are  not  extensive.  For  normal  operation,  the  vehicle  and  sensor 
trailer  are  stored  in  a  garage  or  a  two-door  shipping  container  overnight.  Power  at  the  storage 
site  is  required  to  charge  the  vehicle  batteries  overnight.  Other  batteries  (sensors,  GPS  base 
station,  radios,  etc.)  can  be  charged  off  site  or  in  a  hotel  room.  For  extended  surveys,  an  office 
space  is  procured  for  the  data  preprocessor  and  for  serving  as  an  electronics  diagnosis  and  repair 
area.  The  minimum  survey  crew  consists  of  a  vehicle  driver,  one  other  field  helper  (for  safety 
considerations),  and  a  data  quality  control  (QC)  and  preprocessing  analyst.  As  the  data 
preprocessor  has  time  available,  target  analysis  can  begin  in  the  field.  The  bulk  of  target  analysis 
is  typically  carried  out  later  at  the  analyst’s  office.  The  addition  of  a  fourth  team  member  who 
splits  time  between  the  field,  data  analysis,  and  general  troubleshooting  is  recommended. 
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2.0  INTRODUCTION 


2.1  BACKGROUND 

UXO  detection  and  remediation  is  a  high  priority  triservice  requirement.  As  the  Defense 
Science  Board  wrote  in  2003,  “Today’s  UXO  cleanup  problem  is  massive  in  scale  with  some 
10  million  acres  of  land  involved.  Estimated  cleanup  costs  are  uncertain  but  are  clearly  tens  of 
billions  of  dollars.  This  cost  is  driven  by  the  digging  of  holes  in  which  no  UXOs  are  present. 
The  instruments  used  to  detect  UXOs  (generally  located  underground)  produce  many  false 
alarms— i.e.,  detections  from  scrap  metal  or  other  foreign  or  natural  objects— for  every 
detection  of  a  real  unexploded  munition  found”  [1]. 

There  has  been  considerable  progress  in  the  detection  of  buried  UXO  in  the  last  15  years.  The 
MTADS,  supported  by  ESTCP,  has  demonstrated  that  UXO  items  with  gauges  larger  than  20 
mm  are  typically  detected  to  their  likely  burial  depths  with  location  accuracies  on  the  order  of 
15  cm  [2],  To  reliably  detect  the  smaller  gauge  munitions  in  this  spectrum,  the  MTADS 
EM61  Mkll  or  GEM-3  arrays  should  be  used  rather  than  the  magnetometer  array. 
Discrimination  of  UXO  from  ordnance  fragments  and  other  clutter  remains  a  problem, 
however.  We  have  shown  that  with  careful  mission  planning  and  a  modest  on-site  training 
effort,  an  MTADS  survey/remediation  project  can  achieve  false  alarm  rates  substantially  lower 
than  those  quoted  for  mag-and-flag  surveys.  However,  there  is  still  much  room  for 
improvement  in  discrimination  ability  that  will  result  in  direct  reduction  of  remediation  costs. 

The  Chemistry  Division  of  the  NRL  has  participated  in  several  programs  funded  by  SERDP 
and  ESTCP  whose  goal  has  been  to  enhance  the  discrimination  ability  of  MTADS.  The 
process  has  been  based  on  making  use  of  both  the  location  information  inherent  in  an  item’s 
magnetometry  response  and  the  shape  and  size  of  information  inherent  in  the  response  to  the 
time-domain  electromagnetic  induction  sensors  that  are  part  of  the  baseline  MTADS  in  either  a 
cooperative  or  joint  inversion.  We  have  already  made  significant  progress  toward  our  goal. 
The  algorithms  and  methods  developed  for  ESTCP  involving  analysis  of  data  from  the 
MTADS  EM61  array  were  applied  in  preliminary  form  at  the  JPG-IV  Demonstration  allowing 
us  to  score  as  one  of  the  small  group  of  approaches  that  showed  any  classification  ability  [3]. 
More  recently,  we  have  used  the  methods  at  JPG-V  and  on  a  live  range,  the  Impact  Area  of 
the  Badlands  Bombing  Range,  SD.  In  all  these  demonstrations,  our  classification  ability  has 
been  limited  by  the  information  available  from  the  sensor.  The  EM61  is  a  time-domain 
instrument  with  a  single  gate  or  four  gates  early  in  time  to  sample  the  amplitude  of  the 
decaying  signal.  To  make  further  progress  on  UXO  classification,  a  sensor  with  more 
information  available  is  required. 

By  far  the  best  results  at  JPG-IV  were  obtained  by  Geophex  and  AETC  using  magnetometers 
and  the  GEM-3  frequency-domain  electromagnetic  induction  sensor  [3].  The  GEM-3  sensor  is 
a  frequency-domain  sensor  with  up  to  10  frequencies  available  for  simultaneous  measurement 
of  the  in-phase  and  quadrature  response  of  the  target.  Thus,  in  principle,  there  is  much  more 
information  available  from  a  GEM-3  sensor  for  use  in  classification  decisions.  Unfortunately, 
the  commercial  GEM-3  sensor  is  a  handheld  instrument  with  relatively  slow  data  rates  and  is 
thus  not  very  amenable  to  rapid,  wide  area  surveys.  ESTCP  Project  200033,  Enhanced  UXO 
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Discrimination  Using  Frequency-Domain  Electromagnetic  Induction,  was  funded  to  overcome 
this  limitation  by  integrating  an  array  of  GEM-3  sensors  with  the  MTADS  platform. 

2.2  OBJECTIVES  OF  THE  DEMONSTRATION 

The  original  objective  of  the  demonstration  was  simple:  Demonstrate  the  classification 
performance  of  the  GEM-3  sensor  in  conjunction  with  the  survey  efficiency  of  the  MTADS. 
As  we  undertook  the  first  task  of  the  program,  specification  of  the  sensor  configuration  most 
amenable  to  platform-based  array  use,  we  quickly  became  aware  of  a  significant  noise  issue 
with  the  original  GEM-3  sensor  when  mounted  on  a  cart.  Our  observations  and  subsequent 
modifications  to  the  original  sensor  are  detailed  in  an  earlier  report  [4], 

The  revised  objective  is  only  slightly  more  complicated:  Demonstrate  the  optimum  system  that 
delivers  the  most  classification  performance  while  retaining  acceptable  survey  efficiency. 
Working  around  a  modified  GEM-3  sensor,  we  have  designed  a  three-sensor  array,  which  will 
be  described  in  detail  in  Sections  2.1  and  5.3,  and  demonstrated  it  at  the  Standardized  UXO 
Demonstration  sites  at  APG  and  YPG.  At  each  of  the  sites,  we  surveyed  the  Calibration 
Lanes,  the  Blind  Test  Grid,  and  as  much  of  the  Open  Field  Area  as  was  possible.  For  the 
Blind  Test  Grid  and  the  Open  Field,  our  ranked  target  picks  were  submitted  to  ATC  for 
scoring.  These  scoring  results  are  the  basis  forjudging  the  success  of  the  demonstrations. 

2.3  REGULATORY  DRIVERS 

Stakeholder  acceptance  of  the  use  of  discrimination  techniques  on  real  sites  will  require 
demonstration  that  these  techniques  can  be  deployed  efficiently  and  with  high  probability  of 
discrimination.  The  first  step  in  this  process  is  to  demonstrate  acceptable  perfonnance  on  a  test 
site  such  as  that  at  APG.  After  that  hurdle  has  been  passed,  successful  demonstration  at  a  live  site 
will  facilitate  regulatory  acceptance  of  the  methods. 
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3.0  TECHNOLOGY 


3.1  TECHNOLOGY  DESCRIPTION 

Each  of  the  GEM-3  sensors  in  the  array  sequentially  transmits  a  composite  waveform  made  up  of 
nine  frequencies  logarithmically  spaced  from  90  Hz  to  just  over  20  kHz  for  one  base  period 
(1/30  s).  Thus,  only  three  complete  cycles  of  the  90  Hz  frequency  are  transmitted  while  many 
thousands  of  cycles  of  the  highest  frequency  are  transmitted.  The  transmit  current  drives  both  a 
transmit  coil  and  a  counterwound  bucking  coil.  This  serves  to  set  up  a  “magnetic  cavity”  inside 
the  bucking  coil,  in  which  a  receive  coil  is  placed.  The  current  induced  in  this  receive  coil  by  the 
induced  fields  in  buried  metal  targets  is  detected,  digitized,  and  frequency  resolved  during  the 
two  subsequent  base  periods  while  the  other  array  sensors  are  transmitting.  The  detected  signal 
is  compared  to  the  transmitted  current  and  reported  relative  to  the  transmit  current  (ppm)  as  both 
an  in-phase  and  quadrature  component. 

Based  on  the  lessons  learned  with  the  prototype  array  [4],  we  arrived  at  a  design  for  the 
demonstration  array,  which  is  shown  in  Figure  1.  The  coils  used  in  the  array  are  96  cm  in 
diameter  with  a  larger  number  of  turns.  New,  higher  current  transmitter  electronics  were 
designed  and  constructed  to  drive  the  coils.  The  product  of  these  two  factors  results  in  a  factor 
of  six  to  eight  increase  in  transmit  moment.  The  new  sensor  electronics  bring  other  benefits  as 
well.  We  are  now  able  to  implement  real-time  low-pass  filtering  of  the  induced  current  using  a 
digital  signal  processor  (DSP)  chip  in  the  receive  circuit.  This  reduces  the  absolute  level  of  the 
noise  in  addition  to  the  relative  decrease  obtained  from  the  increased  transmit  moment.  Other 
deployment  measures  have  been  taken  to  reduce  the  induced  noise  in  the  sensor  array.  The  three 
sensors  are  mounted  on  a  rigid  base  attached  to  the  sensor  tray  to  minimize  relative  motion.  We 
have  optimized  the  sensor  cart  for  a  steady  ride.  This  includes  making  the  wheelbase  as  long  as 
possible  and  controlling  tire  pressure.  We  also  deploy  an  array  of  GPS  receivers  to  measure 
orientation  of  the  sensor  platform  as  well  as  position  and  a  six-axis  inertial  measurement  unit 
(IMU). 


Figure  1.  MTADS  GEM-3  array  mounted  on  the  EM  sensor  trailer. 

Note  the  three  GEM  sensors,  the  three  GPS  antennae, 
and  the  IMU  for  platform  motion  measurement. 
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The  sensor  timing  sequence  was  chosen  to  be  non-synchronous  (each  sensor  operating 
independently)  and  sequential.  This  was  done  to  avoid  the  expense  of  developing  new  drive 
electronics  (the  individual  GEM-3  sensors  are  configured  to  run  independently)  and  the  large, 
unbucked  inter-sensor  interference  that  results  from  simultaneous  operation.  Inter-sensor 
sequencing  is  controlled  by  the  custom  electronics  package  mentioned  above. 

Sequential  operation  of  the  sensors  does  not  cause  the  large  reduction  in  array  sampling  rate  that 
might  be  imagined  on  first  thought.  The  baseline  GEM-3  sensor  operates  on  a  1/30  base  period. 
That  is,  each  sensor  acquires  data  for  a  multiple  of  this  base  period.  It  then  takes  approximately 
two  base  periods  to  perform  the  internal  data  reduction  calculations  before  the  results  of  the 
measurement  are  available  to  the  recording  computer.  If  each  of  the  three  sensors  in  this 
sequential  array  is  configured  to  record  for  a  single  base  period,  they  can  then  be  run  near  their 
maximum  rate.  The  effective  sampling  rate  for  the  three  sensor  array  is  ~9.5  Hz,  which  coupled 
with  a  survey  speed  of  ~3  mph,  results  in  a  down-track  sampling  interval  of  ~15  cm.  This  results 
in  an  acceptable  data  density  for  our  analysis  purposes. 

3.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

No  single  method  currently  available  is  the  magic  bullet  of  classification.  We  have  previously 
demonstrated  [2]  that  some  discrimination  is  possible  using  the  standard  MTADS  if  a  small 
training  area  is  investigated  prior  to  data  analysis  on  the  entire  site  and  the  distribution  of 
ordnance  is  limited.  This  discrimination  is  based  primarily  on  fitted  dipole  “size.”  More 
recently  we  have  shown  that  increased  discrimination  can  be  achieved  by  adding  an  extra 
“dimension”  to  the  discrimination — that  of  “shape.”  For  items  of  the  same  induced  magnetic 
dipole,  we  can  discriminate  based  on  the  ratio  of  responses  of  the  items’  three  axes  to  the  EMI 
sensors  in  the  MTADS  suite.  As  we  have  shown  [5],  this  adds  some  discrimination  capability  to 
the  system. 

Unfortunately,  the  EM  methods  previously  demonstrated  are  not  sufficiently  robust  to  the  sensor 
noise,  position  uncertainty,  and  platfonn  motion  inherent  at  a  real  site.  We  have  proposed  that 
some  of  this  observed  drop  in  performance  at  real  sites  can  be  recaptured  using  a  sensor  with 
more  inherent  information  content.  The  GEM-3  sensor  that  is  the  subject  of  this  demonstration 
is  such  a  sensor.  The  depth  performance  of  the  GEM-3  sensors  is  limited  compared  to  the  time- 
domain  EM  sensors  we  have  demonstrated  in  previous  programs  due  to  the  reduced  transmit 
moment  associated  with  the  non-synchronous  transmit  mode. 

Even  with  the  most  optimistic  result,  however,  these  methods  will  not  result  in  a  perfect  system. 
As  stated  many  times,  this  program  is  based  on  the  idea  of  classification  by  shape.  By  definition, 
this  implies  that  clutter  items  that  have  similar  shapes  to  ordnance  will  be  classified  as  ordnance. 
Items  such  as  pipes  and  post  sections  are  representative  of  this  problem.  If  it  is  important  to 
reduce  remediation  costs  to  the  extent  that  these  items  are  not  dug,  other  methods,  possibly 
sensitive  to  composition  or  the  presence  of  explosive  compounds,  will  have  to  be  employed  in 
conjunction  with  those  being  demonstrated  in  this  program. 
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4.0  PERFORMANCE  OBJECTIVES 


There  were  two  primary  objectives  of  this  Demonstration.  The  first  was  to  demonstrate  the 
MTADS  GEM-3  array  (GEMTADS)  on  ranges  away  from  our  home  location  at  Blossom  Point. 
This  provides  a  measure  of  system  reliability  and  ease  of  use  “on  the  road.”  The  second  was  to 
evaluate  the  probability  of  detection  and  classification  ability  of  the  combined  hardware/analysis 
system  in  a  blind  test.  A  summary  of  the  performance  criteria  and  the  resulting  performance  as 
measured  during  the  demonstration  are  given  in  Table  1. 

Table  1.  Performance  objectives  for  the  demonstration. 


Performance 

Objective 

Metric 

Data  Required 

Success 

Criteria 

Results 

Quantitative  Performance  Objectives 

Probability  of 
detection 

Percent  detected  of  seeded 
items 

•  Scoring  reports  from  ATC 
and  Institute  for  Defense 
Analyses  (IDA) 

Pd=0.98 

Yes 

False  alarm  rate 

Number  of  false  targets 
eliminated  at  a  specified 
confidence  level 

•  Scoring  reports  from  ATC 
and  IDA 

Reduction  of 
false  alarms  by 
>40% 

No 

Sensor  array 

SNR 

Design  of  system  such  that  the 
operating  point  for  the  system 
is  well  above  the  system  noise 
floor 

•  Measured  responses  from 
munitions  of  interest 
(signal) 

•  Site  background  signal 
level  (noise) 

SNR  >  10  for 
all  items  of 
interest 

Qualitative  Performance  Objectives 

Reliability 

Number  of  operational  hours 
recorded  per  work  day 

•  Field  logs  of  operational 
hours  per  day 

>  6/day 

Yes 

Ease  of  use 

System  performs  without 
problems  and  work  flow  is 
manageable  for  all  team 
members 

•  Feedback  from  field  team 

Field  crew  comes 
to  work  smiling 

Yes 

Maintenance 

All  required  maintenance  can 
be  performed  on  site  with 
available  materials 

•  Feedback  from  field  team 
on  usability  of  technology 

•  Time  required 

Minimal  impact 
of  any  required 
maintenance  on 
production  rate 

Yes 

4.1  OBJECTIVE:  PROBABILITY  OF  DETECTION 

The  combination  of  sensor  system  capability  and  data  quality  drive  the  probability  of  detection. 
The  collection  of  high  quality  data  with  a  capable  sensor  system  should  lead  to  a  high  probability 
of  detecting  the  munitions  of  interest  at  the  site. 

4.1.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  seed  items  detected  using  the  specified 
detection  methodology  and  threshold. 
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4.1.2  Data  Requirements 


Scoring  reports  for  the  Standardized  UXO  Test  Sites  are  provided  by  both  the  ATC  and  the  IDA 
based  on  the  prioritized  dig  list  provided  by  the  demonstration  team. 

4.1.3  Success  Criteria 

This  objective  will  be  considered  to  be  met  if  at  least  98%  of  the  seeded  items  are  detected. 

4.1.4  Results 

This  objective  was  successfully  met  for  munitions  items  buried  to  theU.S.  Corps  of  Engineers 
(USCOE)  1 1 -times  limit.  The  GEMTADS  surveyed  the  Calibration,  Blind  Grid,  and  Open  Field 
Areas  at  both  the  APG  and  YPG  Standardized  UXO  Technology  Sites.  For  the  Blind  Grid  Test 
Areas,  the  response  stage  results  from  Reference  11  and  13  are  summarized  in  Tables  2  and  3, 
broken  out  by  munitions  size  and  emplacement  depth.  For  the  Open  Field  Areas,  the  response 
stage  results  from  References  12  and  14  are  summarized  in  Tables  4  and  5,  broken  out  by 
munitions  size  and  emplacement  depth.  The  Pd  is  defined  as  the  number  of  response-stage 
detections/number  of  emplaced  munitions  in  the  test  site.  The  Pfp  is  defined  as  the  (number  of 
response  stage  false  positives)/(number  of  emplaced  clutter  items).  The  probability  of 
background  alann  (Pba)  is  defined  as  the  (number  of  response-stage  background  alarms)/(number 
of  empty  grid  locations)  for  the  Blind  Grid.  For  the  Open  Field,  the  denominator  is  an  arbitrary 
constant. 

The  GEMTADS  did  exhibit  the  ability  to  detect  munitions  at  burial  depths  below  the  USCOE 
1 1 -times  limit,  as  is  discussed  further  in  Section  8.  Specifically,  the  Response  Stage  results  are 
shown  in  Figure  17  (APG  Blind  Grid,  Section  8.1.1),  Figure  24  (APG  Open  Field,  Section  8.2.1), 
and  Figure  27  (YPG  Open  Field,  Section  8.3.1)  broken  out  by  munitions  type. 

Table  2.  GEMTADS  Blind  Grid  Test  Area  response  stage  Pd  results. 


Metric 

Overall 

Standard 

Non¬ 

standard 

By  Size 

By  depth  (m) 

Small 

Medium 

Large 

<0.3  m 

0.3  to  <1  m 

>=lm 

LAPG _ 1 

Pd 

0.85 

0.90 

0.85 

1.00 

0.75 

0.80 

1.00 

0.95 

o.oo  ! 

Pd  low  90% 
confidence 

0.81 

0.80 

0.73 

0.91 

0.61 

0.55 

0.95 

0.88 

0.00 

L^pg _ 1 

Pd 

0.90 

0.90 

0.90 

0.95 

0.85 

0.95 

1.00 

0.90 

0.30  [ 

Pd  low  90% 
confidence 

0.85 

0.83 

0.78 

0.86 

0.69 

0.75 

0.95 

0.79 

0.08 
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Table  3.  GEMTADS  Blind  Grid  Test  Area  response  stage  Pfp  and  Pba  results. 


Metric 

Overall 

By  depth  (m) 

<  0.3  m 

0.3  to  <1  m 

>=1  m 

APG 

Pfp 

0.95 

0.95 

0.95 

1.00 

Pf„  low  90%  confidence 

0.90 

0.89 

0.85 

0.63 

Pba 

0.20 

- 

- 

- 

^PG _ 1 

Pfp 

1.00 

1.00 

1.00 

N/A 

Pf„  low  90%  confidence 

0.97 

0.96 

0.92 

- 

Pba 

0.00 

- 

- 

- 

Table  4.  GEMTADS  Open  Field  Test  Area  response  stage  Pd  results. 


Metric 

Overall 

Standard 

Non¬ 

standard 

By  Size 

By  depth  (m) 

Small 

Medium 

BBSS 

<0.3  m 

0.3  to  <1  m 

>=lm\ 

Lapg _ I 

Pd 

0.70 

0.75 

0.65 

0.70 

0.70 

0.80 

0.80 

0.65 

0.50 

Pd  low  90% 
confidence 

0.68 

0.71 

0.60 

0.65 

0.62 

0.71 

0.77 

0.61 

0.44 

Pd  upper  90% 
confidence 

0.74 

0.79 

0.71 

0.75 

0.74 

0.84 

0.86 

0.73 

0.61 

^PG _ 1 

Pd 

0.80 

0.80 

0.80 

0.80 

0.75 

0.90 

0.80 

0.80 

0.50 

Pd  low  90% 
confidence 

0.76 

0.75 

0.75 

0.74 

0.69 

0.83 

0.79 

0.76 

0.38 

Table  5.  GEMTADS  Open  Field  Test  Area  response  stage  PfP  and  Pba  results. 


Metric 

Overall 

By  depth  (m) 

<  0.3  m 

0.3  to  <1  m 

>=1  m 

Lapg 

0.50 

0.45 

0.55 

0.70 

Pf„  low  90%  confidence 

0.48 

0.41 

0.52 

0.50 

Pf„  upper  90%  confidence 

0.52 

0.47 

0.58 

0.84 

0.20 

- 

- 

- 

YPG 

1  * 

0.80 

0.80 

0.80 

PfD  low  90%  confidence 

0.76 

0.76 

0.76 

0.00 

- 

- 

- 

4.2  OBJECTIVE:  FALSE  ALARM  RATE 

A  successful  anomaly  classification  scheme  should  lead  to  a  low  false  alann  rate. 

4.2.1  Metric 

The  false  alarm  rate  is  defined  as  the  number  of  false  alanns  generated  for  some  given  measure, 
such  as  a  survey  area  (#/ha).  The  fractional  reduction  in  false  alarms  produced  by  using  the 
demonstrated  classification  scheme  as  compared  to  using  no  classification  scheme  is  the  metric 
for  success  for  this  objective. 
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4.2.2  Data  Requirements 


Scoring  reports  for  the  Standardized  UXO  Test  Sites  are  provided  by  both  the  ATC  and  the  IDA 
based  on  the  prioritized  dig  list  provided  by  the  demonstration  team. 

4.2.3  Success  Criteria 

A  successful  anomaly  classification  scheme  should  lead  to  a  40%  reduction  in  the  false  alarm 
rate  as  compared  to  classifying  all  detected  anomalies  as  hazardous. 

4.2.4  Results 

This  objective  was  not  successfully  met.  The  GEMTADS  surveyed  the  Calibration,  Blind  Grid, 
and  Open  Field  Areas  at  both  the  APG  and  YPG  Standardized  UXO  Technology  Sites.  The 
results  for  the  Blind  Grid  and  Open  Field  Test  Areas  are  summarized  in  Table  6  and  Table  7, 
taken  from  subsections  of  References  11-14.  Efficiency  (E)  and  false  positive  rejection  rate 
(Rip)  are  used  to  score  discrimination  performance  ability  at  two  specific  operating  points  on  a 
receiver  operating  characteristic  (ROC)  curve — one  at  the  point  where  no  decrease  in  Pd  is 
incurred  and  the  other  at  the  operator-selected  threshold.  Efficiency  is  defined  as  the  fraction  of 
detected  ordnance  correctly  classified  as  ordnance,  and  the  false  positive  rejection  rate  is  defined 
as  the  fraction  of  detected  clutter  correctly  classified  as  clutter. 

Table  6.  GEMTADS  Blind  Grid  Test  Area  efficiency  and  rejection  rates. 


Efficiency 

(E) 

False  Positive 
Rejection  Rate 

APG 

At  operating  point 

0.92 

0.30 

With  no  loss  of  Pd 

1.00 

0.06 

YPG 

At  operating  point 

0.99 

0.15 

With  no  loss  of  Pd 

1.00 

0.01 

Table  7.  GEMTADS  Open  Field  Test  Area  efficiency  and  rejection  rates. 


Efficiency 

(E) 

False  Positive 
Rejection  Rate 

APG 

At  operating  point 

0.78 

0.33 

With  no  loss  of  Pd 

1.00 

0.02 

YPG 

At  operating  point 

0.95 

0.17 

With  no  loss  of  Pd 

1.00 

0.02 

4.3  OBJECTIVE:  ARRAY  SNR 

To  be  successful,  a  data  collection  system  must  be  designed  such  that  at  the  operating  point  for 
the  munitions  of  interest  the  signal  level  is  well  above  the  system  noise  floor. 
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4.3.1  Metric 


The  average  SNR  for  the  munitions  of  interest  is  used  to  evaluate  this  objective. 

4.3.2  Data  Requirements 

To  evaluate  this  metric,  known  exemplar  data  segments  for  the  munitions  of  interest  from  the  site 
and  the  documented  system  background  (noise)  level  are  required. 

4.3.3  Success  Criteria 

The  measured  signal  and  background  levels  should  produce  an  SNR  of  at  least  10  for  all  items  of 
interest. 


4.3.4  Results 


Given  that  the  GEMTADS  array  operates  with  up  to  10  transmit  frequencies  and  records  both  in- 
phase  and  quadrature  responses,  there  is  no  easy  definition  of  SNR  for  this  frequency  domain 
sensor.  Both  the  signal  from  UXO  and  the  noise  from  several  sources  are  a  function  of  both 
frequency  and  in-phase/quadrature.  Also  at  issue  is  the  SNR  level  for  adequate  detection  (>3) 
versus  the  level  for  adequate  discrimination  (>10). 


Figure  2  plots  background  noise  levels  from  various  test  sites  surveyed  by  the  GEM  array.  The 
top  plot  is  the  in-phase  noise  and  the  bottom  plot  is  the  quadrature  noise,  both  as  a  function  of 
transmit  frequency.  The  black  curves  are  from  Blossom  Point,  the  red  curves  from  Aberdeen 
Proving  Ground,  and  the  green  curves  from  Yuma  Proving  Ground.  In-phase  noise  is  roughly 
constant  across  frequencies  but  varies  from  two  to  eight  ppm  between  the  sites.  Quadrature  noise 
increases  at  low  and  high  frequencies  but  is  very  quiet  in  the  mid-frequency  range  (<1  ppm).  The 
various  causes  of  these  noise  behaviors  were  reported  on  in  a  later  study  [6].  From  a  detection 
standpoint,  mid-range  quadrature  signal  levels  of  several  ppm  are  adequate. 
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2.  GEMTADS  background  noise  levels  as  measured 
at  the  three  demonstration  sites. 
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Signal  strength  is  driven  by  the  magnetic  polarization  terms  for  each  UXO  item.  These  terms 
vary  for  both  in-phase  and  quadrature  as  a  function  of  frequency.  For  elongated  UXO,  there  is  a 
set  for  along-axis  response  and  across-axis  response.  Figures  3  and  4  plot  these  responses  for  the 
UXO  items  present  at  the  standardized  test  site  at  APG.  Each  plot  is  for  a  particular  UXO  item. 
There  is  a  large  degree  of  variation  for  all  these  items.  The  red  curves  are  the  quadrature 
response  and  the  blue  curves  the  in-phase  response.  The  solid  curves  and  dashed  lines  are  for  the 
along-axis  and  the  across-axis  responses,  respectively.  For  ferrous  UXO,  the  in-phase  response 
changes  from  negative  to  positive  across  the  frequency  range.  The  quadrature  response  is  always 
positive.  For  large  ordnance,  quadrature  response  peaks  at  lower  frequencies  and  small  ordnance 
at  higher  frequencies.  The  largest  in-phase  response  is  usually  within  a  factor  of  two  to  four  of 
the  largest  quadrature  response.  The  across-axis  response  is  always  weaker  than  the  along-axis 
response.  Therefore,  the  average,  across-axis  quadrature  response  is  considered  a  good  indication 
of  the  weakest  possible  signal  from  an  object.  It  is  therefore  a  reasonable  measure  of  SNR  for  the 
UXO  targets. 

Figures  5  and  6  plot  the  averaged  over  mid-frequencies,  across-axis  quadrature  response  for  the 
APG  ordnance  items  as  a  function  of  depth.  This  is  the  black  curve  in  each  plot.  The  noise  level 
for  this  signal  was  a  fraction  of  1  ppm  at  APG.  The  detection  SNR  level  (>3)  was  roughly  1  ppm 
and  the  dotted  red  curves  indicate  this  limit.  The  symbols  in  these  plots  indicate  how  well  the 
GEM  array  performed  on  the  Blind  Grid  at  APG.  The  red  X’s  are  items  that  were  not  detected. 
The  red  symbols  are  plotted  at  the  depths  from  the  ground  truth  with  the  peak  signal  measured  at 
the  expected  object  location.  In  all  missed  item  cases,  the  black  predicted  curve  falls  below  the  1 
ppm  limit  at  the  ground  truth  depths.  These  undetectable  UXO  items  were  all  buried  at  depths 
greater  than  1 1  times  their  diameter;  most  current  EMI  sensors  would  not  detect  these  items. 
(This  result  is  shown  again  in  Section  8.1.1  and  Figure  17.)  From  a  detection  stand  point,  the 
SNR  levels  of  the  GEM  array  are  adequate. 

Target  discrimination  with  the  GEM-3  array  was  conducted  by  using  a  model-based  inversion  of 
the  polarization  responses  from  the  data  and  matching  these  responses  to  a  library  of  responses 
for  the  expected  UXO  items.  If  the  inverted  parameters  match  a  particular  library  item  well,  the 
object  is  identified  as  this  type  of  UXO.  If  the  parameters  don’t  match  any  library  item  well,  the 
item  would  be  identified  as  clutter.  The  inverted  parameters  were  for  both  in-phase  and 
quadrature  across  the  frequency  range  plotted  in  Figures  3  and  4.  The  library  was  made  up  of  the 
responses  shown  in  these  figures  with  some  added  entries  where  ordnance  orientation  had  an 
effect  on  the  response  (such  as  aluminum  tail  fins  for  an  8 1  mm  nose  down).  Good  inversion  of 
the  polarization  parameters  requires  a  high  SNR  signal — SNR  levels  of  10  or  greater.  For 
quadrature  parameters,  this  criterion  would  be  met  with  a  signal  of  10  ppm  or  greater  on  the  APG 
site.  The  in-phase  noise  levels  were  greater  (-2  ppm)  and  in-phase  signals  of  20  ppm  would  be 
required.  Because  the  library  matching  looks  at  both  in-phase  and  quadrature  across  all 
frequencies,  there  is  no  clear  level  of  signal  strength  that  predetennines  good  discrimination. 
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Figure  3.  In-phase  (blue)  and  quadrature  (red)  response  as  a  function  of  frequency  for 
eight  of  the  APG  UXO  set  and  the  GEMTADS  array. 

Along-  and  across-track  responses  are  shown  as  solid  and  dashed  lines,  respectively. 
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Figure  4.  In-phase  (blue)  and  quadrature  (red)  response  as  a  function  of  frequency  for  five 

of  the  APG  UXO  set  and  the  GEMTADS  array. 

Along-  and  across-track  responses  are  shown  as  solid  and  dashed  lines,  respectively. 
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Depth  Depth 


Figure  5.  GEMTADS  across-axis  quadrature  response,  averaged  over  mid-frequencies  for 
eight  of  the  APG  ordnance  items  as  a  function  of  depth  (black  curve). 

The  detection  SNR  level  is  also  shown  (dotted  red  curves).  The  symbols  indicate  the  GEM-3 

array  performance  on  the  Blind  Grid  at  APG. 
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Depth 

Figure  6.  GEMTADS  across-axis  quadrature  response,  averaged  over  mid-frequencies  for 
five  of  the  APG  ordnance  items  as  a  function  of  depth  (black  curve). 

The  detection  SNR  level  is  also  shown  (dotted  red  curves).  The  symbols  indicate  the  GEM-3 

array  performance  on  the  Blind  Grid  at  APG. 

As  a  rough  indication,  Figures  5  and  6  plot  the  region  of  10-20  ppm  in  gray  as  a  discrimination 
threshold.  Actual  discrimination  performance  on  the  APG  Blind  Grid  is  plotted  by  the  green 
(correctly  indentified  UXO),  blue  (indentified  as  another  UXO),  and  magenta  (UXO  called 
clutter)  symbols.  Many  of  the  ordnance  items  buried  on  the  Blind  Grid  had  signal  strengths  at  or 
below  the  10-20  ppm  level  (70%).  The  general  trend  is  that  these  weaker  UXO  were  incorrectly 
identified.  While  not  shown,  60%  of  the  clutter  had  signal  strengths  of  less  than  20  ppm.  Almost 
all  this  weak  clutter  was  incorrectly  identified  as  UXO.  Most  of  the  strong  signaled  clutter  was 
correctly  flagged.  The  prevalence  of  SNR  levels  of  less  than  10  does  appear  to  be  a  limiting 
factor  in  the  GEM-3  array  data. 

There  are,  however,  a  fair  number  of  exceptions.  The  20  mms  were  correctly  identified  in  the  1 
to  10  ppm  region.  The  81  mms  were  flagged  as  UXO,  but  the  wrong  type.  Most  of  them  were 
identified  as  2.75-inch  warheads,  which  closely  matches  the  81  mm  library  response.  Besides 
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SNR,  another  significant  factor  for  this  site  is  the  large  number  of  items  that  need  to  be  identified 
and  discriminated.  Given  the  combination  of  low  SNR  and  the  large  library  to  compare  fitted 
data  to,  it  is  no  surprise  that  the  discrimination  results  were  so  poor. 

4.4  OBJECTIVE:  RELIABILITY 

One  of  the  main  objectives  of  this  demonstration  is  to  collect  high-quality,  classification  level 
data  at  survey  production  rates.  As  a  base  requirement  to  achieve  this  objective,  the  system  must 
be  capable  of  routinely  collecting  data  for  a  full  work  day. 

4.4.1  Metric 

This  metric  is  simply  the  average  number  of  hours  that  the  system  operated  each  work  day. 

4.4.2  Data  Requirements 

Field  records  will  be  used  to  document  the  demonstration  performance  towards  this  objective. 

4.4.3  Success  Criteria 

Routine  survey  operations  of  6  hours  or  more  per  field  day. 

4.4.4  Results 

This  objective  was  successfully  met.  For  full  field  work  days  (i.e.,  not  including  mobilization 
days),  data  collection  was  under  way  for  at  least  6  hours  per  day.  On  many  days,  8-10  hours  of 
data  collection  occurred. 

4.5  OBJECTIVE:  EASE  OF  USE 

This  objective  represents  an  opportunity  for  all  parties  involved  in  the  data  collection  process, 
especially  the  vehicle  operator,  to  provide  feedback  on  areas  where  the  process  could  be 
improved. 

4.5.1  Metric 

The  system  performs  without  significant  and  unexpected  problems  for  the  field  crew. 
Additionally,  the  tempo  of  the  data  preprocessing  work  flow  is  manageable  for  all  team 
members. 

4.5.2  Data  Requirements 

Discussions  with  the  entire  field  team  and  other  observations  were  used  along  with  field  records. 

4.5.3  Success  Criteria 

A  four-person  team  is  able  to  perform  all  data  collection  and  analysis  tasks  without  a  backlog 
developing.  Additionally,  the  field  team  comes  to  work  each  day  smiling. 
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4.5.4  Results 


This  objective  was  successfully  met.  The  use  of  the  four-person  field  team  provided  sufficient 
personnel  to  allow  for  continuous  operations  without  undo  fatigue  throughout  the  work  day  by 
rotation  of  duties  among  the  field  team. 

4.6  OBJECTIVE:  MAINTENANCE 

Field  survey  working  conditions  are  hard  on  geophysical  equipment,  especially  custom-modified 
or  custom-build  prototype  systems  that  have  not  had  the  benefit  of  extensive  field  testing  and 
hardening.  On-site  maintenance  is  a  requirement.  Having  the  tooling  and  equipment  spares  on 
site  can  shorten  downtimes  due  to  maintenance  or  a  failure  from  days  to  minutes. 

4.6.1  Metric 

All  routine  maintenance  can  be  performed  on  site  with  available  materials. 

4.6.2  Data  Requirements 

Discussions  with  the  entire  field  team  and  other  observations  were  used  along  with  field  records. 

4.6.3  Success  Criteria 

All  routine  maintenance  was  accomplished  on  site,  and  all  required  spare  parts  were  in  the 
inventory. 

4.6.4  Results 

This  objective  was  successfully  met.  An  issue  with  the  strain  relief  system  for  the  GEM-3  cables 
running  from  the  sensors  to  the  tow  vehicles  was  uncovered  at  YPG.  The  cabling  was  repaired 
on  site  to  continue  the  survey.  The  issue  was  resolved  upon  return  to  our  home  facility  and  did 
not  reoccur  for  the  APG  demonstration. 
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5.0  SITE  DESCRIPTION 


The  location  of  the  demonstration  test  sites  were  selected  by  the  principal  investigator  (PI)  of  the 
Standardized  UXO  Technology  Test  Site  Program.  We  demonstrated  this  technology  at  both  the 
APG  and  YPG  sites. 

5.1  SITE  LOCATION  AND  HISTORY 

The  area  of  the  Aberdeen  Test  Site  is  adjacent  to  the  Trench  Warfare  facility  at  the  APG.  The 
specific  site  was  obviously  used  for  a  variety  of  ordnance  tests  over  the  years.  Our  initial 
magnetometer  and  EMI  survey  was  perfonned  after  a  mag-and-flag  survey  of  the  area  identified 
more  than  a  thousand  remaining  anomalies.  These  data  were  used  for  a  final  cleanup  of  the  site 
prior  to  the  establishment  of  the  Standardized  UXO  Technology  Site  and  test  target 
emplacement. 

The  Yuma  Test  Site  is  located  adjacent  to  the  Mine  Test  Area  at  YPG.  The  area  was  reasonably 
clear  before  establishment  of  the  Standardized  UXO  Technology  Site.  Our  initial  survey  found 
far  fewer  residual  anomalies  after  the  initial  clearance  than  was  the  case  for  the  Aberdeen  site. 

5.1.1  Climate  and  Weather 

During  the  last  week  in  September  and  the  first  week  in  October,  the  normal  high  for  Aberdeen 
is  in  the  low  70s  °F  and  the  normal  low  is  in  the  low  50s  °F.  Average  precipitation  is  0.12  inches 
each  day.  In  2003,  temperatures  were  normal  but  precipitation  was  more  than  twice  the  long¬ 
term  average.  In  fact,  there  were  two  tropical  events  in  the  7  days  before  our  survey.  This 
rendered  much  of  the  test  site  too  wet  for  a  survey.  We  returned  to  Aberdeen  in  June  2004  to 
complete  the  survey.  During  this  survey,  temperatures  were  seasonal  and  precipitation  was  low. 
The  Yuma  Demonstration  was  performed  in  November  2003.  During  the  month  of  November, 
the  nonnal  high  is  in  the  low  80s  °F  and  the  normal  low  is  in  the  high  30s  °F.  Average 
precipitation  is  0.2  inches  for  the  month.  The  average  winds  are  4  mph.  During  the  period  of 
our  survey,  temperatures  were  normal  but  we  again  experienced  an  unusual  rain  event.  This 
caused  a  one-day  delay  in  starting  the  survey  but  did  not  affect  our  perfonnance  otherwise. 

5.1.2  Topography 

The  portions  of  the  Aberdeen  site  covered  in  this  demonstration  (the  Calibration  Lanes,  Blind 
Test  Grid,  and  Open  Field)  are  relatively  flat  and  level.  There  are  some  low-lying  areas  in  the 
northwest  portion  of  the  Open  Field  area  that  tend  to  have  standing  water  during  wet  periods  of 
the  year.  We  were  able  to  survey  through  this  standing  water  with  the  MTADS  magnetometer 
system  but  would  not  have  been  able  to  take  the  GEM-3  array  through  water  of  such  depth.  As  it 
turned  out,  the  standing  water  was  much  worse  during  our  first  visit.  In  June  2004,  the  water 
was  again  at  normal  levels. 

The  scenarios  at  the  Yuma  site  that  were  covered  in  this  demonstration  (the  Calibration  Lanes, 
Blind  Test  Grid,  and  Open  Field)  are  also  relatively  flat  and  level.  Due  to  the  presence  of  washes 
and  scattered  trees  on  the  Open  Field  site,  we  were  not  able  to  cover  every  square  meter  of  the 
site  with  the  vehicular  towed  system.  We  surveyed  along  the  edge  of  the  washes  on  both  sides 
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and  then  down  in  them  as  was  feasible.  The  coverage  we  achieved  in  the  Open  Field  site  was 
representative  of  vehicle-towed  systems. 

5.1.3  Site  Maps  and  Photographs 

An  aerial  photograph  of  the  Aberdeen  test  site  is  shown  in  Figure  7.  The  various  scenarios  are 
outlined  and  numbered.  The  scenarios  of  concern  here  are  the  Calibration  Lanes  (1)  and  Blind 
Test  Grid  (2)  outlined  in  blue  and  the  Open  Field  (3)  outlined  in  yellow.  We  did  not  survey 
either  the  mogul  (4)  or  wooded  (5)  areas. 


Figure  7.  Aerial  photograph  of  the  Aberdeen  Test  Site  with  the  various  scenarios  outlined. 

A  corresponding  aerial  photograph  of  the  Yuma  test  site  is  shown  in  Figure  8.  The  various 
scenarios  are  outlined.  The  scenarios  of  concern  here  are  the  Calibration  Lanes,  the  Blind  Test 
Grid,  and  the  Open  Field.  We  did  not  survey  either  the  mogul  or  Yuma  Desert  Brush  areas. 
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Figure  8.  Aerial  photograph  of  the  Yuma  Test  Site  with  the  various  scenarios  outlined. 


5.2  MUNITIONS  CONTAMINATION 


The  APG  and  YPG  Standardized  UXO  Technology  Sites  were  seeded  with  a  variety  of 
munitions  and  submunitions  by  the  Pis  of  the  sites  from  the  ATC  Standardized  Target 
Repository  [7].  The  included  munitions  types  span  a  large  range  of  sizes  and  material 
compositions.  They  include  small  projectiles  (20  mm  M55,  25  mm  M792,  37  mm  M63M1, 
40  mm  Midi,  and  57  mm  M86),  submunitions  (M42,  BDU-26,  and  BDU-28),  medium  mortars 
(60  mm  M49A3  and  81  mm  M374),  and  large  projectiles  (105  mm  M456  HEAT,  105  mm  M60, 
and  155  mm  M483A1).  A  variety  of  clutter  has  also  been  seeded  within  the  sites. 
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6.0  TEST  DESIGN 


6.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 


The  demonstration  was  designed  to  be  executed  in  two  stages.  During  the  first  stage  the 
GEMTADS  sensor  array,  as  demonstrated,  was  characterized  with  respect  to  the  items  of 
interest.  Measurements  of  example  munition  articles  of  interest  were  made  to  generate  library 
signatures  for  use  during  the  data  analysis  and  classification  process. 

The  second  stage  of  the  demonstration  consisted  of  surveys  of  the  APG  and  YPG  Standardized 
UXO  Technology  Sites  using  the  GEMTADS.  The  Calibration  and  Blind  Grid  areas  at  each  site 
were  surveyed.  For  each  site,  the  surveyable  portion  of  the  Open  Field  scenario  was  surveyed  as 
well.  Each  data  set  was  then  inverted  using  the  data  analysis  methodology  discussed  in  Section 
7.3,  and  estimated  target  parameters  determined.  These  target  parameters  were  then  used  to 
classify  each  detected  anomaly  as  described  in  Section  7.4.  The  results  are  discussed  at  length  in 
Reference  [8]. 


The  schedule  of  field  testing  activities  is  provided  in  Figure  9  as  a  Gantt  chart. 
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Figure  9.  Schedule  of  field  testing  activities. 
6.2  SITE  PREPARATION 


As  the  APG  and  YPG  Standardized  UXO  Technology  Sites  are  established  sites  with  logistics  in 
place,  no  additional  site  preparation  was  required. 

6.3  SYSTEMS  SPECIFICATION 


This  demonstration  was  conducted  using  the  NRL  MTADS  tow  vehicle  and  subsystems.  The 
tow  vehicle  and  each  subsystem  are  described  further  in  the  following  sections. 

6.3.1  MTADS  Tow  Vehicle 


The  MTADS  has  been  developed  by  the  NRL  Chemistry  Division  with  support  from  ESTCP. 
The  MTADS  hardware  consists  of  a  low-magnetic-signature  vehicle  that  is  used  to  tow  the 
different  sensor  arrays  over  large  areas  (10-25  acres/day)  to  detect  buried  UXO.  The  MTADS 
tow  vehicle  and  GEM-3  array  are  shown  in  Figure  10. 
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Figure  10.  MTADS  GEM-3  array  in  operation  pulled  by  the  MTADS  tow  vehicle. 


6.3.2  GEM-3  (GEMTADS)  Array 

As  discussed  in  Section  3.1,  the  MTADS  GEM-3  array  consists  of  three,  96-cm  diameter  GEM-3 
sensors  (Geophex,  Ltd.)  in  a  triangular  configuration  with  two  sensors  across  the  front  of  the 
array  and  one  centered  in  the  rear.  The  nominal  ride  height  of  the  sensors  is  33.5  cm  above  the 
ground.  The  roughly  2-m  square  array  is  shown  schematically  in  Figure  11.  Figures  1  and  10 
show  the  configured  array  being  pulled  by  the  MTADS  tow  vehicle.  The  sensors  have  been 
false-colored  (red,  green,  blue)  in  Figure  1  to  match  the  color  scheme  used  in  the  other  figures  in 
this  report  and  in  the  Data  Acquisition  (DAQ)  system  software  display.  The  array  is  mounted  on 
a  rigid  support  that  is  attached  to  the  MTADS  electromagnetic  (EM)  trailer  using  non-metallic 
fasteners. 

Individual  sensors  in  the  array  are  located  using  a  three-receiver  real  time  kinematic  (RTK)  GPS 
system  shown  schematically  in  Figure  11  [9].  The  three-receiver  configuration  extends  the 
concept  of  RTK  operations  from  that  of  a  fixed  base  station  and  a  moving  rover  to  moving  base 
stations  and  moving  rovers.  The  lead  GPS  antenna  (and  receiver,  MB  1)  receive  corrections  from 
the  fixed  base  station  at  1  Hz.  This  corrected  position  is  reported  at  10-20  Hz  using  a  vendor- 
specific  National  Marine  Electronics  Association  (NMEA)  NMEA-0183  message  format 
(PTNL,GGK  or  GGK).  The  MB1  receiver  also  operates  as  a  “moving  base,”  transmitting 
corrections  (by  serial  cable)  to  the  next  GPS  receiver  (MB2),  which  uses  the  corrections  to 
operate  in  RTK  mode. 

A  vector  (AVR1,  heading  [yaw],  angle  [pitch],  and  range)  between  the  two  antennae  is  reported 
at  10  Hz  using  a  vendor-specific  NMEA-0183  message  format  (PTNL,AVR  or  AVR).  MB2  also 
provides  “moving  base”  corrections  to  the  third  GPS  antenna  (MR),  and  a  second  vector  (AVR2) 
is  reported  at  10  Hz.  All  GPS  measurements  are  recorded  at  full  RTK  precision,  ~2-5  cm.  All 
sensor  readings  are  referenced  to  the  GPS  1-PPS  output  to  fully  take  advantage  of  the  precision 
of  the  GPS  measurements.  An  IMU  is  also  included  on  the  sensor  array  to  provide 
complementary  platform  orientation  information.  The  IMU  is  a  Crossbow  VG600  running  at 
approximately  83  Hz. 

A  close-up  view  of  the  sensor  platform  is  shown  in  Figure  1,  which  shows  the  three  GPS 
antennae  and  the  IMU  (black  box  under  the  aft  port  GPS  antenna).  The  airborne  adjunct  of  the 
MTADS,  the  Airborne  Multisensor  Towed  Array  Detection  System  (AMTADS)  uses  a  similar 
configuration  with  two  GPS  antennae/receivers  to  provide  the  yaw  and  roll  angles  of  the  sensor 
boom  and  pitch  from  the  IMU  [10]. 
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Figure  11.  MTADS  EM  trailer  with  approximate  locations  of  GPS 
and  IMU  equipment  indicated. 

The  colored  circles  represent  the  GEM-3  sensors  of  the  GEMTADS  array. 


The  standard  GEM-3  sensor  drive  electronics  have  been  modified  to  produce  a  substantially 
higher  transmit  moment  for  this  array.  Each  individual  sensor  can  transmit  a  composite 
waveform  of  one  to  10  frequencies  in  the  frequency  range  of  30  to  20,010  Hz  with  a  base  period 
of  1/30  s.  For  this  survey,  a  composite  transmitter  waveform  of  nine  frequencies  log-spaced 
from  90  to  20,010  Hz  is  used.  Two  additional  base  periods  are  required  for  signal  deconvolution 
and  to  output  the  response  from  each  sensor.  The  array  can  therefore  operate  continuously  with 
one  sensor  actively  transmitting  while  the  other  two  sensors  are  processing  data  at  any  given 
time.  Allowing  for  a  short  coil  settling  time  between  the  transmissions  from  each  sensor,  an 
effective  array  sampling  rate  of  just  over  9  Hz  is  achieved.  Sequential  transmitter  operation  also 
alleviates  the  need  for  the  orthogonal  survey  mode  employed  for  the  EM61  Midi  array.  Coupled 
with  our  standard  survey  speed  of  3  mph,  the  result  is  a  down- track  sampling  spacing  of  ~15  cm 
with  a  cross-track  spacing  of  50  cm.  An  interleaved  survey  pattern  is  used  to  decrease  the  cross¬ 
track  spacing  to  25  cm  as  depicted  in  Figure  12. 


The  GEM-3  sensors  are  controlled  by  a  custom  electronics  package  designed  and  built  by 
Geophex,  Ltd.  It  is  mounted  in  an  equipment  rack  in  the  MTADS  tow  vehicle  as  shown  in 
Figure  13.  Overall  control  of  data  collection  is  accomplished  with  a  custom  version  of  the 
standard  GEM-3  sensor  control  software,  WinGem2KArr,  running  under  Windows  2000  on  our 
data  acquisition  computer.  An  example  of  the  working  screen  of  this  program  is  shown  in  Figure 
14.  This  software  package  logs  the  data  from  the  GEM-3  sensors,  the  three  GPS  NMEA 
sentences,  the  time  of  the  GPS  l-PPS  pulse,  the  GPS  Universal  Coordinated  Time  (UTC)  time 
stamp,  and  the  IMU  data  in  separate  files  with  a  common  base  survey  name.  The  data  are 
periodically  transferred  to  the  data  analyst  for  immediate  QC  checks  and  for  further  processing. 
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Figure  12.  Schematic  of  interleaved  survey  pattern  for  GEMTADS  surveys. 

The  sensors  are  depicted  as  colored  circles.  The  large  crosshatched  sections  indicate  the  path  of 
the  tow  vehicle  tires.  The  outer  extents  of  the  swath  of  the  EM  trailer  tires  are  represented  by  the 
narrow  crosshatching.  The  tan  bars  represent  areas  where  two  tire  tracks  are  collocated. 
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6.4  DATA  COLLECTION 

6.4.1  Scale  of  Demonstration 


The  GEMTADS  array  was  deployed  to  the  APG  and  YPG  Standardized  UXO  Technology  Sites 
in  Fall  2003  and  Spring  2004  for  APG,  and  Winter  2003  for  YPG.  The  areas  of  the  surveyed 
portions  of  the  two  Standardized  UXO  Technology  Sites  and  the  coverage  achieved  by 
GEMTADS  array  for  each  area  are  detailed  in  Table  8.  Each  data  set  was  then  inverted  using  the 
data  analysis  methodology  discussed  in  Section  7.3,  and  estimated  target  parameters  detennined. 
These  target  parameters  were  then  used  to  classify  each  detected  anomaly  as  described  in 
Section  6.5. 


Table  8.  Survey  coverage  of  GEMTADS  array  at  the  two  test  sites. 


Site 

Scenario 

Area  (ha) 

GEMTADS 
Array  Coverage 
(ha) 

Fraction 

Aberdeen 

Calibration  Lanes 

0.12 

0.12 

100% 

Blind  Grid 

0.19 

0.19 

100% 

Open  Field 

5.54 

4.89 

88% 

Yuma 

Calibration  Lanes 

0.11 

0.11 

100% 

Blind  Grid 

0.17 

0.17 

100% 

Open  Field 

6.23 

5.92 

95% 

6.4.2  Sample  Density 

Based  on  our  standard  survey  speed  of  3  mph,  a  down-track  sampling  spacing  of  ~15  cm,  and  a 
cross-track  spacing  of  50  cm,  the  expected  data  density  is  14  points  per  m2.  With  the  interleaved 
survey  pattern,  this  doubles  to  28  points  per  m  .  See  Section  6.3.2  for  further  details. 

6.4.3  Quality  Checks 

The  following  procedure  constitutes  a  typical  startup  for  the  MTADS  system  for  both  initial 
startup  and  as  daily  system  evaluations.  The  RTK  GPS  base  station  receiver  and  radio  link  are 
established  on  a  previously  established  control  point.  The  validity  of  the  control  point  location 
will  be  verified  using  the  MTADS  man-portable  RTK  GPS  rover  receiver  to  occupy  one  or  more 
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of  the  established  control  points  using  the  control  point  occupied  by  the  GPS  base  station  as  a 
reference. 

The  standard  performance  checks  include  three  types  of  measurements.  Initial  system  startup,  at 
the  beginning  of  field  work  and  again  each  morning,  consists  of  three  measurements.  First, 
quiet,  static  data  are  collected  for  a  period  of  15-20  minutes  with  all  systems  powered  up  and 
warmed  up  (typically  30  minutes).  Next,  two  calibration  items,  a  4-inch  diameter  aluminum  (Al) 
sphere  and  a  ferrite  rod  bundle,  are  placed  a  standard  distance  above  the  center  of  each  sensor 
coil  several  times  in  sequence  to  verify  the  response  of  each  sensor  to  each  object.  The  system  is 
stationary  for  this  data  collection.  Finally,  a  systems  timing  check  using  a  fixed-position  wire  or 
chain  placed  on  the  ground  is  conducted.  At  the  discretion  of  the  PI,  the  timing  check  may  be 
repeated  in  the  middle  of  the  survey  day.  At  the  discretion  of  the  PI,  the  timing  check  and  the  Al 
sphere  and  ferrite  measurements  may  be  repeated  at  the  end  of  the  day. 

The  data  quality  checks  conducted  on  each  data  set  are  discussed  in  Section  7.1. 

6.4.4  Data  Summary 

The  primary  performance  metrics  of  this  demonstration  are  the  detection  and  discrimination 
performance  of  the  GEMTADS  array  at  the  two  Standardized  UXO  Technology  Demonstration 
Test  Sites.  These  results  are  provided  by  the  site  managers  after  detection  and  discrimination 
picks  are  submitted  [11-14,  12,  13,  14].  The  ground  truth  of  these  sites  is  held  by  the  Pis  and 
results  are  only  reported  in  aggregate. 

6.5  VALIDATION 

With  the  exception  of  the  Calibration  Grid,  the  ground  truth  for  the  Standardized  sites  is  held 
back  from  individual  technology  demonstrators  to  preserve  the  utility  of  the  Blind  Grid  and  Open 
Field  Areas.  Results  from  the  Blind  Grid  and  the  Open  Field  Area  were  submitted  to  ATC  for 
performance  evaluation.  Scoring  results  have  been  received  and  are  available  [11-14].  A 
summary  of  the  results  is  given  in  Section  8. 
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7.0  DATA  ANALYSIS  AND  PRODUCTS 


7.1  PREPROCESSING 

The  processing  steps  necessary  to  consolidate  the  individual  raw  data  fdes  recorded  by  the  data 
acquisition  computer  (refer  to  Section  6.3.2)  to  a  product  feasible  for  analysis  are  coarsely 
outlined  in  the  block  diagram  of  Figure  15. 

Corrections  based  on  recent  or  same  day  ferrite  measurements  are  first  applied  to  each  of  the 
three  GEM-3  data  files.  In  theory,  the  frequency  response  to  ferrite  should  be  flat  and  negative 
for  the  in-phase,  and  zero  for  the  quadrature.  Amplitude  and  phase  corrections  based  on  this  are 
therefore  applied  to  correct  for  and  equalize  the  response  of  all  three  GEM-3  sensors.  In 
addition,  a  running  median  window  is  subtracted  from  each  of  the  GEM-3  time  series  (on  a  per 
frequency  basis)  to  mitigate  sensor  drift — a  problem  especially  at  the  higher  frequencies.  A 
window  width  of  500  points  (~50  s)  was  chosen  for  both  the  APG  and  YPG  Open  Field  GEM-3 
data. 

As  described  in  Section  6.3.2,  the  GPS  data  files  contain  NMEA  GGK  sentences  yielding  the 
position  of  the  master  antenna  (MB1  in  Figure  11)  at  20  Hz,  as  well  as  AVR  sentences  yielding 
the  vectors  to  the  other  two  antennae  (represented  by  the  red  and  blue  arrows  in  Figure  1 1)  at  10 
Hz.  These  vectors  are  defined  by  their  magnitudes  (range)  and  two  angles  (yaw  and  tilt),  with 
the  yaw  angle  defined  relative  to  true  north.  By  first  redefining  the  yaw  angle  to  be  relative  to 
grid  north,  the  along-platform  and  cross-platform  vectors  are  subsequently  computed  from  the 
two  AVR  vectors  allowing  the  pitch  and  roll  angles  of  the  platform  to  be  readily  determined. 
Because  all  sensors  are  mounted  on  the  same  rigid  board,  the  orientation  of  all  the  sensors  is  now 
completely  determined. 

GPS  GEM  IMU 


Synchronization 


I 

Sensor  Location  (X,  Y,  Z)  &  Sensor  Attitude  (Yaw,  Pitch,  Roll) 
of  each  Sensor  Reading  (In-phase  &  Quadrature  I  9  frequencies) 

@  sampling  rate  of  -10  Hz 

Figure  15.  A  schematic  diagram  showing  the  integration  of  the  GEM  and  navigation  data 

for  eventual  analysis. 
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To  perform  the  final  integration  of  data,  all  sensors  need  to  be  synchronized.  As  discussed  in 
Section  6.3.2,  this  is  mostly  taken  care  of  in  the  data  acquisition  phase  by  referencing  all  data  to 
the  GPS  1-PPS,  with  an  added  sensor  latency  determined  by  driving  the  array  back  and  forth 
over  a  metal  pipe.  Since  this  latter  latency  tends  to  change  unpredictably  over  time,  it  is 
necessary  to  perfonn  such  a  measurement  on  a  continual  basis.  On  implementation  of  this 
latency,  the  location  and  orientation — a  combination  of  the  IMU  and  GPS  pitch  and  roll  (Section 
6.3.2) — of  each  GEM  sensor  at  a  given  time  is  matched  to  a  given  spectrum. 

7.2  TARGET  SELECTION  FOR  DETECTION 


The  process  of  anomaly  identification,  or  detection,  from  the  survey  data  was  done  using  the  Qavg 
quantity,  the  average  of  the  quadrature  response  for  the  middle  five  frequencies. 


Qav e 


Z(Q  210  Hz  +  QsiOHz  +  Q\21,0Hz  +  Ql6lOHz  +  0543O//z) 


We  chose  this  metric  because  of  the  lower  noise  in  the  quadrature  response  and  the  good  signal 
in  the  mid  frequencies  for  the  objects  of  interest.  The  performance  of  this  metric  on  the  APG 
and  YPG  data  sets  is  discussed  in  Sections  8.1.1  (APG  Blind  Grid),  8.2.1  (APG  Open  Field),  and 
7.3.1  (YPG  Open  Field)  respectively. 

7.3  PARAMETER  ESTIMATES 

The  located  demedianed  GEMTADS  data  (position,  orientation,  and  nine  data  pairs  (in-phase 
and  quadrature  response  for  nine  transmit  frequencies))  surrounding  the  center  of  each  selected 
anomaly  are  extracted  and  submitted  to  a  solver  engine  developed  by  Science  Applications 
International  Corporation  (SAIC)  for  the  analysis  of  the  GEMTADS  data  using  a  standard  dipole 
model.  The  solver  algorithm  finds  six  fitted  parameters — x,  y,  z,  (p,  6,  ijf — where  x,  y,  z  are  the 
target  coordinates  (m)  and  <p,  0,  ijf  are  the  Euler  angles  (deg)  of  the  model  output.  The 
associated  best-fit  betas  (set  of  three  Ps  at  each  frequency  for  in-phase  and  quadrature),  chi- 
squared  (%“),  and  coherence  are  also  found  as  part  of  the  inversion.  The  betas  are  the  principal 
components  of  the  induced  magnetization  tensor.  The  and  coherence  are  measures  of  the 
quality  of  the  inversion  result  with  respect  to  the  original  source  data.  Follow-on  fits  are  made 
while  constraining  the  three  Ps  to  match  appropriate  given  values  from  a  library  containing 
values  for  items  including  the  items  of  interest.  The  associated  yj  and  coherence  values  are 
found  for  the  fit  to  each  library  item,  and  the  ratios  of  (constrained  library  fit)/(unconstrained 
library  fit)  for  both  %  and  coherence  are  calculated. 

For  each  anomaly  processed  in  the  described  manner,  all  fitted  results  are  reported,  including 
position  ( x,y,z ),  orientation  (three  angles),  polarizability  spectra,  %J  error,  the  correlation 
coefficient  between  model  and  measured  data,  and  a  signal  strength  metric.  Both  unconstrained 
and  library-constrained  inversions  are  performed,  assuming  that  representative  spectra  are 
observed  for  certain  classes  of  objects  (such  as  the  UXO  targets  of  interest  or  unique  clutter 
classes).  The  size  of  an  equivalent  sphere  is  also  estimated  from  the  Ps  via  established 
parametric  models. 
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7.4  CLASSIFIER  AND  TRAINING 


The  prescription  for  discrimination  using  GEMTADS  is  to  compare  the  measured  response  of  a 
target  to  each  of  a  set  of  library  response  functions  in  turn  and  to  determine  which  library  item 
results  in  the  closest  match  to  the  target.  If  the  match  is  good  enough,  we  can  declare  the  target 
as  being  that  member  of  the  library.  To  quantify  the  “goodness-of-fit”  to  this,  we  compute  the  X 
for  the  each  match.  Further  details  on  the  system  perfonnance  are  given  in  Section  8.1.2. 

7.5  DATA  PRODUCTS 

The  primary  data  products  generated  are  the  ranked  dig  lists  submitted  to  the  site  managers  for 
scoring.  Additionally,  the  archival  data  sets,  stored  individually  by  site  and  by  area  are  provided 
for  future  data  analysis/classification  efforts. 
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8.0  PERFORMANCE  ASSESSMENT 


The  project  objective  was  to  demonstrate  the  optimum  system  built  around  the  Geophex  GEM-3 
EMI  sensor  that  delivers  the  most  classification  performance  while  retaining  acceptable  survey 
efficiency.  A  three-sensor  array  system  was  designed  around  a  modified  GEM-3  sensor.  The 
system  was  built  and  characterized  and  then  demonstrated  at  the  Standardized  UXO 
Demonstration  sites  at  APG  and  YPG.  At  each  of  the  sites,  the  Calibration  Lanes,  the  Blind  Test 
Grid,  and  as  much  of  the  Open  Field  Area  as  was  possible  were  surveyed.  For  the  Blind  Test 
Grid  and  the  Open  Field,  the  ranked  target  picks  were  submitted  to  ATC  for  scoring.  These 
scoring  results  are  the  basis  for  characterizing  the  success  of  the  demonstrations  and  the 
performance  of  the  array.  Portions  of  Reference  8  are  reproduced  here  to  summarize  the 
performance  of  the  system. 

8.1  ABERDEEN  PROVING  GROUND  BLIND  GRID 

8.1.1  Response  Stage 

The  first  stage  of  scoring  at  the  Test  Sites  is  the  Response  Stage  where  anomalies  are  identified 
or  detected.  For  this,  we  use  the  Qavg  quantity,  as  discussed  in  Section  7.2.  A  Qavg  plot  for  the 
APG  Blind  Grid  is  shown  in  Figure  16.  The  400  cells  in  the  Blind  Grid  are  marked  with  white 
squares  in  Figure  16.  A  summary  of  the  GEMTADS  array  detection  performance  is  given  in 
Table  9. 


Table  9.  Summary  of  detection  performance  at  the  APG  Blind  Grid. 


Cell  Contents 

Number  of 
Cells 

Number 

Correct 

Number 

Incorrect 

Single  ordnance  item 

84 

73 

11 

Ordnance  item  with  clutter 

7 

7 

0 

Single  clutter  item 

95 

91 

4 

Two  clutter  items 

8 

8 

0 

“Empty” 

206 

174 

32 

Total 

400 

The  32  cells  reported  as  “Empty”  but  for  which  we  made  a  declaration  require  some  discussion. 
Only  12  of  these  false  positives  showed  signal  in  the  GEM  array  survey  only.  Seven  of  these 
cells  had  a  detection  by  the  GEM  array,  the  EM61  HH,  and  the  magnetometer  array.  Ten  had  a 
detection  by  the  GEM  array  and  the  EM6 1  HH,  and  three  had  a  detection  by  the  GEM  array  and 
the  magnetometer  array.  An  example  of  this  is  cell  B8,  which  is  highlighted  in  Figure  16.  It  is 
difficult  to  understand  the  observed  signal  unless  there  is  some  inadvertent  metal  in  this  cell. 
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Figure  16.  Qavg  anomaly  image  map  of  the  APG  Blind  Grid. 


An  indication  of  the  depth  performance  of  the  system  is  shown  in  Figure  17.  The  detected  items 
are  shown  as  black  triangles  and  the  missed  items  are  shown  as  red  crosses.  The  reference  line 
corresponds  to  a  depth  of  1 1 -times  the  item  diameter.  As  can  be  seen,  the  GEM  array  is  capable 
of  detecting  targets  down  to  and  below  1 1  times  their  diameter  at  this  site. 
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Figure  17.  Qavg  detection  performance  as  a  function  of  depth  at  the  APG  Blind  Grid. 

The  response  stage  data  are  plotted  in  Figures  18  and  19  in  the  manner  of  the  Standardized  Test 
Site  scoring  reports.  Figure  18  shows  cumulative  ordnance  count  versus  cumulative  clutter 
count.  Since  the  targets  are  ordered  by  signal  amplitude  at  the  response  stage,  it  is  no  surprise 
that  this  plot  is  essentially  along  the  diagonal. 


clutter  count 

Figure  18.  Response  stage  results  showing  cumulative  ordnance 
count  versus  cumulative  clutter. 

A  better  measure  of  system  capability  is  shown  in  Figure  19,  which  plots  cumulative  occupied 
cells  versus  adjusted  cumulative  blank  cells.  Cells  such  as  B8,  which  obviously  contain  buried 
metal,  were  excluded  from  the  blank  count. 
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adjusted  blank  count 

Figure  19.  Response  stage  performance  showing  cumulative  occupied  cell  count  plotted 

versus  adjusted  cumulative  blank  cell  count. 

8.1.2  Discrimination  Stage 

Discrimination  using  GEMTADS  is  accomplished  by  comparing  the  measured  response  of  a 
target  to  each  of  a  set  of  library  response  functions  in  turn  and  determining  which  library  item 
results  in  the  closest  match  to  the  target.  If  the  match  is  good  enough,  we  can  declare  the  target 
as  being  that  member  of  the  library.  To  quantify  the  goodness-of-fit,  we  compute  the  %“  for  each 
match.  For  the  APG  Blind  Grid,  three  different  methods  of  discrimination  based  on  the  x  were 
evaluated. 

The  first  method  is  one  based  on  the  weighing  of  the  x  by  the  signal  amplitude.  If  the  signal 
level  for  the  various  targets  differs  by  a  large  amount  (some  targets  are  quite  shallow  and  some 
very  deep)  the  computed  x  can  be  strongly  affected  by  the  signal  amplitude.  To  test  this 
possibility,  we  computed  the  %2  for  the  best  match  weighting  the  data  by  the  usual  1/rms2  (where 
the  root  mean  square  [nns]  deviation  is  detennined  from  areas  between  targets)  and  by  l/(rms  + 
0.01  x  signal)  in  an  attempt  to  reduce  the  influence  of  signal  amplitude  on  the  computed  x  •  The 
X  calculated  with  the  signal-based  weighting  was  used  for  our  declarations  at  the  APG  Blind 
Grid.  Based  on  the  results  from  the  Calibration  Lanes  (which  was  ah  we  had  available  at  the 
time),  we  established  an  x  threshold  of  0.01  for  the  ordnance/clutter  decisions.  This  is  a  little 
less  than  three  standard  deviations  above  the  ordnance  mean.  The  reported  Discrimination 
Response  Factor  was  just  the  inverse  of  the  x  ■ 

The  second  method  assumes  that  this  strong  variation  of  X  with  signal  amplitude  arises  from  the 
bouncing  motion  of  the  sensor  array  as  it  traverses  the  rough  field.  Over  a  high-signal  target, 
small  variations  in  z  result  in  relatively  large  variations  in  signal  as  compared  to  over  a  deep, 

low-signal  target.  In  this  case,  we  can  model  the  bouncing  noise  by  (K  *  signal),  and  the  correct 

2*2 

weighting  would  be  l/(rms"  +  (K*  signal)").  Based  on  data  collected  on  the  Blossom  Point  Test 
Field,  at  values  of  K  around  0.3,  the  scaling  of  x  with  signal  amplitude  seems  to  flatten  out. 
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Scaling  the  weights  by  the  signal  improves  the  performance  of  the  discrimination  but  is  not  very 
practical  as  the  scaling  coefficient  is  determined  after  the  fact.  For  the  later  YPG  and  APG  Open 
Field  demonstrations,  we  employed  another  method  to  mitigate  the  effects  of  bouncing  noise. 
Each  target  was  fit  using  a  full,  unconstrained  3-P  model  as  well  as  the  library  model.  The  ratio 
of  the  y2  for  these  two  methods,  which  eliminates  the  dependence  on  signal  amplitude,  should 
approach  1  if  the  item  is  in  the  library. 

The  ROC  curves  for  the  application  of  these  three  discrimination  methods  to  the  APG  Blind  Grid 
are  shown  in  Figures  20  through  22.  The  standard  y~  weighting  (Figure  20)  and  the  modified 
weighting  with  “bounce  noise”  added  (Figure  21)  result  in  curves  that  vary  little  from  the  chance 
diagonal.  There  are  fewer  items  in  Figure  21  than  in  Figure  20.  The  original  submission  to  APG 
required  that  a  discrimination  score  be  included  for  all  cells,  even  those  below  our  detection 
threshold.  We  arbitrarily  assigned  these  cells  a  low  discrimination  score.  The  y  with  “bouncing 
noise”  analysis  was  applied  only  to  cells  in  which  we  declared  a  detection. 


clutter  count 

Figure  20.  ROC  curve  for  the  y2  weighting  applied  to  the  APG  Blind  Grid  as  shown  in  the 
left-hand  side  of  Figures  25  and  26  of  Reference  8. 
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Figure  21.  ROC  curve  for  the  case  of  /2  weighting  with  an  estimate  of  “bouncing  noise” 

included  applied  to  the  APB  Blind  Grid. 

The  %2  ratio  method  (Figure  22)  does  show  some  promise.  Notice,  however,  that  the  curve  in 
Figure  22  includes  even  fewer  ordnance  and  clutter  items  than  in  Figure  21.  The  %“  ratio  method 
requires  two  different  inversions  to  converge  to  sensible  results  in  order  to  calculate  the  ratio.  As 
the  SNR  decreases,  this  becomes  an  increasingly  difficult  hurdle.  Library  methods  such  as  this 
can  work  well  when  the  expected  targets  are  well  defined  but  can  provide  inappropriate  results 
when  a  munitions  item  not  in  the  library  is  encountered. 


Figure  22.  ROC  curve  for  the  %2  ratio  method  applied  to  the  APG  Blind  Grid. 

8.2  ABERDEEN  PROVING  GROUND  OPEN  FIELD 

Selected  results  from  our  surveys  at  the  Open  Field  at  the  APG  Standardized  Test  Site  have  been 
provided  to  us  by  analysts  at  the  IDA.  These  results  are  summarized  graphically  in  the  following 
sections. 
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8.2.1  Response  Stage 


Response  stage  data  from  the  Open  Field  scenario  at  the  APG  Standardized  Test  Site  is  shown  in 
Figure  23  as  a  plot  of  probability  of  detection  versus  normalized  background  alarm  rate.  There 
are  two  analysis  models  shown  on  the  plot.  The  first,  the  red  line,  corresponds  to  considering 
only  those  targets  that  were  covered  by  the  survey  and  are  not  within  2  m  of  another  target.  The 
analysis  corresponding  to  the  blue  line  retains  those  limitations  and  also  excludes  those  targets 
deeper  than  1 1 -times  their  diameter.  We  showed  in  Figure  17  that  the  GEM-3  array  is  able  to 
detect  down  to  depths  of  at  least  1 1 -times  a  target’s  diameter  (and  below  for  small  to  medium 
targets).  Response  stage  results  broken  out  by  item  type  are  shown  in  Figure  24.  In  this  figure, 
the  depth  of  100%  detection  is  denoted  by  the  blue  bar  and  the  depth  of  maximum  detection  is 
shown  as  the  horizontal  line.  For  a  number  of  the  items,  1 05 -mm  HEAT  for  example,  these  two 
depths  are  the  same.  For  the  majority  of  the  items,  the  maximum  depth  of  detection  is  below  the 
depth  of  100%  detection. 


1.0 

1 

0.8 

_ i - 

1 

0.6 

- 1 

1 

0.4 

0.2 

-  surveyed,  2-m  exclusion 

less  than  1  lx  depth 
-  surveyed,  2-m  exclusion 

0.0 

I  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1  1 

0.00  0.05  0.10  0.15  0.20  0.25 

normalized  background  alarm  rate 

Figure  23.  Detection  performance  at  the  APG  Open  Field  scenario. 

The  red  line  is  derived  considering  only  targets  that  were  covered  in  the  survey  and  are  not 
within  2  m  of  another  target.  The  blue  line  retains  those  criteria  and  also  excludes  targets  deeper 

than  1 1 -times  their  diameter. 
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Figure  24.  Response  stage  results  for  the  APG  Open  Field  scenario 
broken  out  by  target  type. 


8.2.2  Discrimination  Stage 

Discrimination  stage  performance  at  the  APG  Open  Field  using  the  same  two  analysis  models  is 
shown  in  Figure  25.  As  above,  the  exclusion  of  items  at  depths  below  11 -times  their  diameter 
(presumably  lower  signal-to-noise  [S/N]  anomalies)  improves  the  discrimination  performance 
obtained. 


Figure  25.  Discrimination  performance  at  the  APG  Open  Field  scenario. 

The  red  line  is  derived  considering  only  targets  that  were  covered  in  the  survey  and  are  not 
within  2  m  of  another  target.  The  blue  line  retains  those  criteria  and  also  excludes  targets  deeper 

than  1 1 -times  their  diameter. 
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8.3  YUMA  PROVING  GROUND  OPEN  FIELD 


Selected  results  from  our  surveys  at  the  Open  Field  at  the  YPG  Standardized  Test  Site  have  been 
provided  to  us  by  analysts  at  the  IDA.  These  results  are  summarized  graphically  in  the  following 
sections. 

8.3.1  Response  Stage 

Response  stage  results  for  the  YPG  Open  Field  scenario  are  shown  in  Figures  26  and  27.  As  for 
APG,  they  are  analyzed  by  excluding  first  items  that  were  not  covered  by  the  survey  or  are 
within  2  m  of  another  item,  then  retaining  those  exclusions  and  further  excluding  items  deeper 
than  1 1 -times  their  diameter.  Notice  that  the  background  alarm  rates  in  Figure  26  are  more  than 
a  factor  of  two  smaller  than  the  corresponding  results  from  Aberdeen.  Although  the  Yuma  site  is 
more  geologically  active  than  Aberdeen,  it  is  smoother  so  there  were  fewer  false  alarms  due  to 
platform  bouncing  over  deep  ruts.  Detection  depths  at  Yuma  are,  in  general,  in  line  with  those 
obtained  at  Aberdeen.  Note  however,  that  a  shallow  bomb  was  apparently  missed  resulting  in  an 
unusual  plot  for  that  target  type. 


1.0 

- ' - 1 - ' - 1 - 

- 1 - ' - 1 - ' - 1 - 

1 - 

i - 

0.8 

0.6 

- 

"O 

CL 

0.4 

■ 

0.2 

—  surveyed,  2-m  exclusion 
less  than  1  lx  depth 

—  surveyed,  2-m  exclusion 

0.0 

_ 1 _ 1 _ 1 _ 1 _ 

_ 1 _ 1 _ 1 _ 1 _ 1 _ 

0.000  0.002  0.004  0.006  0.008  0.010 

normalized  background  alarm  rate 


Figure  26.  Detection  performance  at  the  YPG  Open  Field  scenario. 

The  red  line  is  derived  considering  only  targets  that  were  covered  in  the  survey  and  are  not 
within  2  m  of  another  target.  The  blue  line  retains  those  criteria  and  also  excludes  targets  deeper 

than  1 1 -times  their  diameter. 
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Figure  27.  Response  stage  results  for  the  YPG  Open  Field  scenario 
broken  out  by  target  type. 


8.3.2  Discrimination  Stage 

Discrimination  Stage  results  from  the  YPG  Open  Field  are  shown  in  Figure  28.  As  before, 
exclusion  of  items  that  are  deeper  than  1 1 -times  their  diameter  improves  performance,  which  is 
better  on  the  whole  than  that  observed  at  Aberdeen.  As  with  the  response  stage,  this  is  likely  due 
to  the  lower  platform  motion  noise  observed  at  the  Yuma  site. 


Figure  28.  Discrimination  performance  at  the  YPG  Open  Field  scenario. 

The  red  line  is  derived  considering  only  targets  that  were  covered  in  the  survey  and  are  not 
within  2  m  of  another  target.  The  blue  line  retains  those  criteria  and  also  excludes  targets  deeper 

than  1 1 -times  their  diameter. 
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9.0  COST  ASSESSMENT 


9.1  COST  MODEL 

The  costs  for  a  50-acre  survey  using  the  GEMTADS  array  in  discrimination  mode  are  detailed 
in  Table  10.  These  surveys  require  interleaved  sampling,  as  was  practiced  in  the  two 
Standardized  Test  Site  surveys.  The  coverage  using  this  collection  methodology  is  ~5 
acres/7.5  hour  survey  day.  Thus,  a  50-acre  survey  requires  two  weeks  in  the  field.  All  costs 
are  estimated  in  FY  2006  dollars. 


Table  10.  Summary  of  Costs  for  a  50-acre  GEM  array  survey. 


Cost  Category 

Sub-Category 

Cost 

Sub-Total 

Mobilization  costs 

$63,500 

Preliminary  site  visit 

$8000 

Test  plan  preparation 

$10,000 

Equipment  prep  and  packing/unpacking 

$12,000 

Rental  trailer  and  transportation 

$17,500 

Analysts  setup 

$10,000 

Real-time  travel  for  4  personnel 

$6000 

Logistics 
(if  required) 

$13,600 

Establish  GPS  control  points 

$4000 

Office  and  garage  trailers,  portable  toilets 

$2350 

Generator/fuel/electrician 

$4000 

Materials 

$3250 

Operating  costs 
(2-week  survey) 

$59,500 

Supervisor 

$12,000 

On-site  analyst 

$9500 

Vehicle  operator 

$6100 

Field  technician 

$7500 

Per  diem  (4  personnel  x  14  days  x  $150) 

$8400 

Rental  vehicles 

$4000 

Equipment  repair 

$12,000 

Analysis  &  reporting 

$25,000 

Total  cost 

$161,600 

9.2  COST  DRIVERS 

Two  factors  were  expected  to  be  strong  drivers  of  cost  for  this  technology  as  demonstrated.  The 
first  is  the  acreage  that  can  be  surveyed  per  day.  The  actual  acreage  covered  sets  the  amount  of 
data  collected  per  day.  Higher  productivity  in  data  collection  equates  to  more  total  acreage 
covered  for  a  given  period  of  time  in  the  field.  The  time  required  for  selecting  and  analyzing 
individual  anomalies  can  be  significantly  higher  than  other,  traditional  methods  and  could 
become  a  cost  driver  due  to  the  time  involvement.  The  thoughtful  use  of  available  automation 
techniques  for  individual  anomaly  analysis  with  operator  QC  support  can  moderate  this  effect. 

9.3  COST  BENEFIT 

The  main  benefit  to  using  a  UXO  classification  process  is  cost-related.  The  ability  to  reduce  the 
number  of  nonhazardous  items  that  have  to  be  dug  or  dug  as  hazardous  directly  reduces  the  cost 
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of  a  remediation  effort.  The  additional  infonnation  for  anomaly  classification  provided  by  the 
GEMTADS  array  provides  additional  information  for  the  purposes  of  anomaly  classification.  If 
there  is  buy-in  from  the  stakeholders  to  use  these  techniques,  this  infonnation  can  be  used  to 
reduce  costs. 
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10.0  IMPLEMENTATION  ISSUES 


Implementation  of  the  methods  used  in  this  demonstration  requires  additional  survey  time 
compared  to  a  minimum-effort  detection  survey.  We  have  shown  that  the  MTADS  can  detect  all 
UXO  of  gauges  40  mm  or  larger  with  a  total-field  magnetometry  survey  alone.  For  ordnance 
target  sets  that  include  60-  and  81 -mm  mortars  at  depths  of  0.75  to  1  m  and/or  20-  and  30-mm 
submunitions,  an  overlapping  EMI  survey  is  required  to  get  a  high  detection  probability. 
Compared  to  the  magnetometer  array,  we  use  two  interleaved  GEM-3  array  surveys  to  ensure 
sufficient  data  density  over  each  target  to  get  a  reliable  fit  to  our  models.  This  increases  the 
survey  hours  onsite  although  it  does  not  impact  the  mobilization  and  data  analysis  costs.  In 
many  cases,  the  extra  survey  costs  are  equivalent  only  to  the  cost  of  digging  one  or  two 
additional  targets  per  acre. 

Discrimination-quality  data  collection  requires  either  interleaved  sampling  (in  the  case  of  the 
GEM-3  array)  or  more  rigorously  surveying  in  two  orthogonal  directions  (in  the  case  of  the 
EM61  array).  This  requires  another  factor  of  two  penalty  on  the  EM  systems  for  an  overall 
productivity  reduction  of  a  factor  of  four.  The  sensor  transmitters  used  for  the  GEM-3  sensor  in 
the  demonstrated  array  are  fired  in  a  sequential  pattern  unlike  the  MTADS  EM61  array  where 
the  sensor  transmitters  are  synchronized  for  maximum  transmit  moment.  Because  the  EM61 
array  transmitters  are  synchronized,  the  interleaved  survey  passes  must  be  orthogonal  as  well  to 
ensure  both  sufficient  data  density  and  sufficient  target  illumination.  The  sequential  transmitter 
pattern  of  the  GEM-3  array  removes  the  orthogonality  constraint,  which  can  be  onerous  on  some 
sites,  and  terrain  such  as  on  a  steep  hill  face. 

The  GEMTADS  array  is  a  vehicular-towed  array  with  the  operating  limitations  of  a  towed 
system.  Support  requirements  are  not  extensive.  For  normal  operation,  the  vehicle  and  sensor 
trailer  are  stored  in  a  garage  or  similar  space  overnight;  this  has  often  been  a  two-door  Conex 
trailer.  AC  power  is  required  at  the  storage  site  to  charge  the  vehicle  batteries  overnight.  Other 
batteries  (sensors,  GPS  base  station,  radios,  etc.)  can  be  charged  off  site  or  in  a  hotel  room  but,  of 
course,  it  is  most  convenient  to  charge  them  at  the  storage  site.  For  extended  surveys,  an  office 
space  for  the  data  preprocessor  and  to  serve  as  an  electronics  diagnosis  and  repair  area  is 
suggested.  All  of  this  support  was  provided  at  the  Standardized  Test  Sites,  requiring  no 
additional  action  for  the  demonstration. 

The  minimum  survey  crew  consists  of  a  vehicle  driver,  one  other  field  helper  (for  safety 
considerations),  and  a  data  QC  and  preprocessing  analyst.  The  surveys  described  in  this  report 
included  an  additional  member  of  the  team  who  split  time  between  the  field,  data  preprocessing, 
and  general  troubleshooting.  As  the  data  preprocessor  has  time  available,  target  analysis  can 
begin  in  the  field.  The  bulk  of  target  analysis  is  carried  out  later  at  the  analyst’s  office. 
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